Translation compensation in optical image stabilization (OIS)

ABSTRACT

Techniques described herein can address these and other issues by synchronizing the positioning of an adjustable lens in a camera assembly with the capture of an image frame by the image sensor and optimizing the position of the adjustable lens to reduce the amount of blur caused by translation of the camera assembly along a direction along the optical axis over the course of a frame. More specifically, techniques provide for moving the lens to a plurality of optimized positions, relative to the image sensor, over the course of a frame, to reduce motion blur in an image due to translation of the camera assembly in a direction of along the optical axis during the frame. Some embodiments may provide for “tight” synchronization in cases where the plurality of optimized positions are based on a time-dependent function that takes into account when each row of the image sensor is being exposed over the course of the frame.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is continuation of U.S. Non-Provisional applicationSer. No. 16/295,146, filed on Mar. 7, 2019, and titled “TranslationCompensation In Optical Image Stabilization (OIS),” which is herebyincorporated by reference in its entirety for all purposes.

BACKGROUND

Optical Image Stability (OIS) techniques improve the performance ofcamera assemblies by counteracting image blur due to camera unsteadinessor jitter and/or to compensate for rolling shutter distortions duringimage capture. This may be particularly important for camera assembliesincorporated into handheld devices such as mobile phones and tabletcomputing devices for still or video images. OIS techniques utilize oneor more actuators coupled with a camera lens and/or an image sensor ofthe camera assembly that translate, tilt, and/or rotate the lens and/orsensor relative to the camera assembly in at least one of the pitch,roll, and yaw directions. As such, OIS techniques may largely orcompletely compensate for effects of camera motion, including rotation(that may be measured gyroscopically, for example) and translation (thatmay be measured by an accelerometer, for example) and/or rolling shuttereffects.

Implementing these OIS techniques in a camera assembly, however, is notwithout its trade-offs. Due to power, size, cost, and/or other limitingconstraints, some camera systems may have limited OIS functionality.Techniques for improving OIS functionality in these limited systems,therefore, can increase the user experience of such limited-capabilitycamera systems.

SUMMARY

Techniques described herein can address these and other issues bysynchronizing the positioning of an adjustable lens in a camera assemblywith the capture of an image frame by the image sensor and optimizingthe position of the adjustable lens to reduce the amount of blur causedby translation of the camera assembly along a direction along theoptical axis over the course of a frame. More specifically, techniquesprovide for moving the lens to a plurality of optimized positions,relative to the image sensor, over the course of a frame, to reducemotion blur in an image due to translation of the camera assembly in adirection of along the optical axis during the frame. Some embodimentsmay provide for “tight” synchronization in cases where the plurality ofoptimized positions are based on a time-dependent function that takesinto account when each row of the image sensor is being exposed over thecourse of the frame.

An example method of providing synchronized optical image stabilizationin a camera assembly having an adjustable lens, according to thedescription, includes, for an image sensor of the camera assembly withsensor elements disposed in rows oriented in a direction along a firstaxis and columns oriented in a direction along a second axis, obtaininginformation indicative of the beginning of a frame of the image sensor,wherein the frame comprises a period of time having an exposure periodfollowed by a readout period in which exposure values are read from thesensor elements row by row, and obtaining at least one distance valueindicative of at least one distance of the image sensor to at least oneobject viewable by the image sensor. The method also includes, for eachtime of a plurality of times during the frame, obtaining respectivemovement data indicative of translation of the camera assembly in adirection along a third axis orthogonal to the first axis and the secondaxis, the translation corresponding to the respective time, determininga respective value of a time-dependent function based at least in parton the information indicative of the beginning of the frame, wherein therespective value of the time-dependent function is dependent on therespective time in relation to the frame, and causing the adjustablelens to be moved to a respective position relative to the image sensorduring the frame, wherein the respective position in a direction alongthe first axis, in a direction along the second axis, or both, is atleast partially based on the respective movement data, the respectivevalue of the time-dependent function, and the at least one distancevalue.

An example camera assembly with optical image stabilization, accordingto the description, comprises a controller configured to becommunicatively coupled with an image sensor with sensor elementsdisposed in rows oriented in a direction along a first axis and columnsoriented in a direction along a second axis, one or more actuatorsconfigured to move an adjustable lens that focuses light onto the imagesensor, and a motion sensor. The controller is configured to obtaininformation indicative of the beginning of a frame of the image sensor,wherein the frame comprises a period of time having an exposure periodfollowed by a readout period in which exposure values are read from thesensor elements row by row. The controller is also configured to obtainat least one distance value indicative of at least one distance of theimage sensor to at least one object viewable by the image sensor, and,for each time of a plurality of times during the frame, obtainrespective movement data, from the motion sensor, indicative oftranslation of the camera assembly in a direction along a third axisorthogonal to the first axis and the second axis, the translationcorresponding to the respective time, determine a respective value of atime-dependent function based at least in part on the informationindicative of the beginning of the frame, wherein the respective valueof the time-dependent function is dependent on the respective time inrelation to the frame, and cause the one or more actuators to move theadjustable lens to a respective position relative to the image sensorduring the frame, wherein the respective position in a direction alongthe first axis, in a direction along the second axis, or both, is atleast partially based on the respective movement data, the respectivevalue of the time-dependent function, and the at least one distancevalue.

An example apparatus, according to the description, comprises means forobtaining, for an image sensor of the apparatus with sensor elementsdisposed in rows oriented in a direction along a first axis and columnsoriented in a direction along a second axis, information indicative ofthe beginning of a frame of the image sensor, wherein the framecomprises a period of time having an exposure period followed by areadout period in which exposure values are read from the sensorelements row by row, and means for obtaining at least one distance valueindicative of at least one distance of the image sensor to at least oneobject viewable by the image sensor. The example apparatus furthercomprises means for, for each time of a plurality of times during theframe, obtaining respective movement data indicative of translation ofthe apparatus in a direction along a third axis orthogonal to the firstaxis and the second axis, the translation corresponding to therespective time, determining a respective value of a time-dependentfunction based at least in part on the information indicative of thebeginning of the frame, wherein the respective value of thetime-dependent function is dependent on the respective time in relationto the frame, and causing the adjustable lens to be moved to arespective position relative to the image sensor during the frame,wherein the respective position in a direction along the first axis, ina direction along the second axis, or both, is at least partially basedon the respective movement data, the respective value of thetime-dependent function, and the at least one distance value.

An example non-transitory computer-readable medium, according to thedescription, has instructions embedded thereon for providingsynchronized optical image stabilization in a camera assembly having anadjustable lens. The instructions, when executed by one or moreprocessing units, cause the one or more processing units to, for animage sensor of the camera assembly with sensor elements disposed inrows oriented in a direction along a first axis and columns oriented ina direction along a second axis, obtain information indicative of thebeginning of a frame of the image sensor, wherein the frame comprises aperiod of time having an exposure period followed by a readout period inwhich exposure values are read from the sensor elements row by row,obtain at least one distance value indicative of at least one distanceof the image sensor to at least one object viewable by the image sensor,and, for each time of a plurality of times during the frame, obtainrespective movement data indicative of translation of the cameraassembly in a direction along a third axis orthogonal to the first axisand the second axis, the translation corresponding to the respectivetime, determine a respective value of a time-dependent function based atleast in part on the information indicative of the beginning of theframe, wherein the respective value of the time-dependent function isdependent on the respective time in relation to the frame, and cause anadjustable lens to be moved to a respective position relative to theimage sensor during the frame, wherein the respective position in adirection along the first axis, in a direction along the second axis, orboth, is at least partially based on the respective movement data, therespective value of the time-dependent function, and the at least onedistance value.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an electronic device with a camera assembly,according to an embodiment;

FIG. 2 is a simplified cross-sectional view of a camera assembly,according to an embodiment;

FIG. 3 is a graph illustrating example motion blur in an image due totranslation movement in a direction along the optical axis during imagecapture;

FIG. 4 is a graph illustrating example OIS motion blur adjustment for anarea of interest in an image, according to an embodiment;

FIG. 5 is a frame graph showing how the image sensor may capture animage frame, according to an embodiment;

FIG. 6 is a graph illustrating OIS motion blur compensation for opticalaxis translation in an image having two areas of interest, according toan embodiment;

FIG. 7 is a frame graph, similar to FIG. 5, illustrating exposure andreadout of sensor rows over the course of a frame;

FIG. 8 is a graph illustrating how the OIS motion blur compensation foroptical axis translation performed in FIG. 7 can reduce or eliminatemotion blur in a direction along the y-axis of the resulting image,according to an embodiment;

FIGS. 9 and 10 are frame graphs illustrating successively-capturedframes;

FIG. 11 is a flow diagram of a method of providing synchronized OIS in acamera assembly having an adjustable lens, according to an embodiment.

Like reference symbols in the various drawings indicate like elements,in accordance with certain example implementations. In addition,multiple instances of an element may be indicated by following a firstnumber for the element with a letter or a hyphen and a second number.For example, multiple instances of an element 110 may be indicated as110-1, 110-2, 110-3 etc. or as 110 a, 110 b, 110 c, etc. When referringto such an element using only the first number, any instance of theelement is to be understood (e.g., element 110 in the previous examplewould refer to elements 110-1, 110-2, and 110-3 or to elements 110 a,110 b, and 110 c).

DETAILED DESCRIPTION

Several illustrative embodiments will now be described with respect tothe accompanying drawings, which form a part hereof. While particularembodiments, in which one or more aspects of the disclosure may beimplemented, are described below, other embodiments may be used andvarious modifications may be made without departing from the scope ofthe disclosure or the spirit of the appended claims. Additionally, thedescribed embodiments may be implemented in any device, apparatus, orsystem that incorporates a camera, such as mobile telephones, multimediaInternet enabled cellular telephones, mobile television receivers,wireless devices, smartphones, wearable devices, personal dataassistants (PDAs), wireless electronic mail receivers, hand-held orportable computers, tablets, cameras, digital media players (such as MP3players), camcorders, game consoles, wrist watches, mobile healthdevices, and so forth, as well as larger systems (e.g., automobiles,smart buildings, etc.) that incorporates such electronics.

It can be noted that, as used herein, the terms “image frame” and“frame” refer to a period of time in which an image is captured from animage sensor (e.g., a CMOS sensor). This includes an exposure period inwhich photosensitive sensor elements of the image sensor are configuredto sense light and produce a corresponding value (e.g., a voltage)indicative of and amount of light sensed during the exposure period, anda readout period in which exposure values of the sensor elements areread and processed by image processing circuitry. For rolling-shuttercameras, the exposure values of the sensor elements are read out a rowat a time (row-by-row) over the course of the readout period.

It can be additionally noted that, as a matter of convention, an imagesensor as described in the embodiments herein is described with regardto a coordinate frame having mutually-orthogonal x, y, and z axes, whererows of image sensor are aligned with an x-axis of the image sensor andcolumns are aligned with a y-axis of the image sensor. The optical axisof the image sensor therefore corresponds to the z-axis. The axes of thecamera assembly comprising the image sensor. (This coordinate frame isillustrated in applicable figures, to help illustrate how the variousfigures are oriented within the coordinate frame.) The coordinate systemfor an electronic device incorporating the camera assembly may be thesame, or may be different, depending on convention. The person ofordinary skill in the art will recognize that values of magnitude and/orangles utilized within this coordinate frame may be positive ornegative, depending on the convention used. As such, variables disclosedin mathematical formulas, equations, and/or expressions provided hereinmay be positive or negative, depending on convention.

FIG. 1 is a diagram of an electronic device 100 with a camera assembly,according to an embodiment. A person of ordinary skill in the art willappreciate that, although a mobile phone is pictured as the electronicdevice 100, a variety of alternative devices may also have a cameraassembly and may therefore utilize techniques for optical imagestabilization (OIS) described herein. In this example, the coordinatesystem for the electronic device 100 is different than the coordinatesystem of the camera assembly, as shown by device axes 104 and cameraaxes 106. In particular the x-axis 107 and y-axis 108 of the device axes104 are respectively aligned with the y-axis 130 and x-axis 120 of thecamera axes 108. For convention, and as previously mentioned,embodiments described herein use a coordinate frame with respect to thecamera aperture 110 as illustrated by camera axes 108, with mutuallyorthogonal axes (the x-axis 120, the y-axis 130, and the z-axis 140).The camera aperture 110 provides an opening for light to illuminate theimage sensor of a camera assembly (not shown) incorporated into theelectronic device 100, where the image sensor utilizes the coordinateframe of the camera axes. Accordingly, embodiments shown in the figuresand described below utilize the camera axes 108 (not the device axes106).

In the orientation illustrated in FIG. 1, (i.e., “landscape”orientation, where rows of the image sensor along the x-axis 120 arehorizontal), y-axis rotation 150 represents a change in the yaw of theelectronic device 100, and x-axis rotation 160 represents a change inthe pitch of the electronic device 100. (In a “portrait” orientation,where rows of the image sensor along the x-axis 120 are vertical, pitchwould correspond to y-axis rotation 150 and yaw would correspond tox-axis rotation 160.) Traditional 2-axis OIS systems may provide forblur compensation due to x-axis rotation 150 and y-axis rotation 150.Traditional 4-axis is OIS systems may further provide blur compensationfor y translation 170 (i.e., translation in a direction of the y-axis130) and x translation 180 (i.e., translation in a direction of thex-axis 120).

The techniques disclosed herein provide for blur reduction due to z-axis(optical axis) translation 190. Because these techniques utilize lensdisplacement in a direction along the x-axis 120 and/or y-axis 130, theymay be implemented by traditional 2-axis OIS systems and/or 4-axis OISsystems that are capable of such x- and y-axis lens displacement.Moreover, the techniques herein are additive, and may therefore be usedsimultaneously with traditional 2-axis OIS and/or 4-axis OIS, therebyeffectively providing 3-axis OIS and/or 5-axis OIS, respectively. Anexample of a camera assembly capable of implementing the blur reductiontechniques disclosed herein is provided in FIG. 2

FIG. 2 is a simplified cross-sectional view of a camera assembly 200,according to an embodiment. Some or all components of the cameraassembly 200 may be integrated into a larger electronic device, such asthe electronic device 100 of FIG. 1. In the illustrated implementation,a camera assembly 200 includes a lens 210 and an image stabilizingarrangement (ISA) 220. The ISA 220 may include an image sensor 222optically coupled with the lens 210 and an actuator(s) 224 mechanicallycoupled with the lens 210. Although only on actuator 224 is illustrated,one or more actuator(s) 224, which in some implementations may comprisea voice coil motor (VCM) actuator, may be coupled with the lens 210 andpositioned to shift (or translate) the lens 210 in a direction along thex-axis 120 and/or y-axis (not shown, but orthogonal to the xz plane).Thus, actuator(s) 224 may be configured to move the lens 210 to variouslocations in the xy plane, orthogonal to the optical axis 211. (Aspreviously noted, the optical axis 211 is the z-axis 140.) In someembodiments, one or more actuators additionally or alternatively may beused to translate the image sensor 222 with respect to the lens 210 in asimilar manner. (Other embodiments may be additionally capable oftranslating the lens 210 and/or image sensor 222 along the optical axis211 and/or rotating the lens 210 and/or image sensor 222, depending onthe type of OIS implemented.) In some embodiments, the camera assembly200 may include an adjustable aperture (not shown), which can be used toadjust an amount of light to which the image sensor 222 is exposed.

The camera assembly 200 further includes a controller 230,communicatively coupled with the ISA 220 and at least one motion sensor240 for measuring camera motion. The controller may comprise aprocessing unit (e.g., a microprocessor, digital signal processor (DSP),application-specific integrated circuit (ASIC), a programmable gatearray (PGA) or similarly programmable circuitry, which may be configuredto interact with the other components of the camera assembly 200 asdescribed herein. As such, the controller 230, may be configured toexecute software stored in a non-transitory computer-readable medium.

The motion sensor 240 may comprise a gyroscope, an accelerometer, and/orany other type of motion sensor capable of determining rotational and/ortranslational movement of the camera assembly 200 (e.g., which reflectrotational and/or translational movement of an electronic device 100into which the camera assembly 200 is integrated, as shown in FIG. 1).In some embodiments, a camera assembly 200 may not have a motion sensor240 dedicated to OIS. In such embodiments, the controller 230 may obtainmotion data from other components of an electronic device into which thecamera assembly 200 is integrated.

OIS is used to reduce blurring and other effects associated withmovement of the camera assembly 200 during the time the image sensor 222is capturing an image (e.g., the exposure period of a frame). Putgenerally, the motion sensor 240 (which may be configured to capturemotion data at a certain frequency) provides motion data to thecontroller 230 regarding movement of the camera assembly 200. Thecontroller 230 causes the actuator(s) 224 to move the lens 210 in the xyplane in a manner that compensates for the movement and minimizes blurin an image captured by the image sensor 222 caused by the motion of thecamera assembly. Thus, an amount of OIS lens shift is proportional tothe amount of movement.

For a camera assembly 200 that moves the lens 210 in a plane parallel tothe image sensor 222 (e.g., the xy plane), there is a limitation ofnon-uniform image quality improvement. This is because lens 210 and/orimage sensor 222 shift in the xy plane can only cause uniform imageshift by so many pixels. This can be a particular challenge when themotion blur is due to translation in a direction along the optical axis211, when good compensation would require non-uniform shift.

FIG. 3 is a graph illustrating motion blur in an image 300, due totranslation movement in a direction along the optical axis during imagecapture. Here, the image 300 comprises an array of pixels having (as anexample) 1080 rows and 1920 columns. (Alternative embodiments may havedifferent resolutions and/or aspect ratios.) Pixels in the image 300correspond to sensor elements of the image sensor 222 disposed incorresponding rows and columns in the xy plane. Thus, image rows arealigned with the x-axis 120 and columns are aligned with the y-axis 130,as shown.

As shown, blurring near the periphery of the image is larger thanblurring near the center of the image. (Peripheral blur 310 is largerthan central blur 320.) Accordingly, an OIS correction due to lens shiftin the xy plane, which corrects blurring uniformly, may undercompensatefor blurring near the periphery of the image and/or overcompensate forblurring near the center of the image 300. However, if an “area ofinterest” within the image is determined, OIS lens shift may be modifiedto compensate for roll in a manner that moves the clearest area of theimage 300 from the center of the image 300 to the area of interest.(This modification of lens shift is also called “lens shiftadjustment.”)

It can be noted that the blurring effect illustrated in image 300, isfor an image of a target at uniform distance from the image sensor 222.Blurring can vary, depending on speed of translation and the distance ofthe target. More specifically, blurring due to parallax, which may beestimated as:

$\begin{matrix}{{Parallax} = \frac{Translation}{{Distance}\mspace{14mu}{to}\mspace{14mu}{Target}}} & (1)\end{matrix}$

Traditional 4-axis OIS typically compensates for parallax due totranslation along the x- and/or y-axis using a common distance for allfeatures, as if the target were flat. Moreover, traditional 4-axis OISalso typically only performs this compensation for close-up photography,where a small distance to the target leads to a large parallax.

However, as described in further detail herein, techniques for adjustingOIS lens shift to provide blur reduction due to optical axis translationas described herein can optimize blur reduction for non-flat (3-D)target support and/or take into account blur due to not only a smalldistance to the target, but also large translation (high-speedmovement). Such blur reduction may reduce blur at one or more locationswithin an image 300 by moving the sharpest portion of the image 300(known as the “vanishing point”) from the center to an “area ofinterest” centered elsewhere in the image 300, or provide optimal OISlens shift for each row of the image 300, based “tight” synchronizationwith a frame to determine when rows of the image sensor 222 are exposedover the course of a frame. Additional details regarding theseembodiments are provided herein below.

FIG. 4 is a graph illustrating motion blur in an image 400, which,similar to FIG. 3, is due to translation movement in a direction alongthe optical axis during image capture. Here, however, OIS lens shift hasbeen adjusted during the course of the image capture such that thevanishing point (with very little blur due to the Z translation) islocated at an area of interest 410, rather than at the center of theimage 400. The area of interest may be determined automatically by theelectronic device 100 (e.g., centered at a location of a face or otherdetected objects, which may be determined from one or more preliminaryimages) or manually by a user of the electronic device.

For an area of interest 410 centered at a particular row (Row) andcolumn (Col), the x and y components of the stretch/compression speed atthe area of interest can be calculated as:

$\begin{matrix}{{{{AI}_{Xspeed}(t)} = {\frac{{ForwardTranslation}Spee{d(t)}}{{Distance}({AI})}*\left( {{Col} - {MidColumn}} \right)}},} & (2)\end{matrix}$for the x component, and

$\begin{matrix}{{{{AI}_{Yspeed}(t)} = {\frac{{ForwardTranslation}Spee{d(t)}}{{Distance}({AI})}*\left( {{Rol} - {MidRow}} \right)}},} & (3)\end{matrix}$for the y component. Here, the terms MidRow and MidColumn respectivelyrefer to the row and column numbers of the middle row and middle columnof the image 400. (In the image 400 of FIG. 4, for example, which has1080 rows and 1920 columns, MidRow would equal 540 and MidColumn wouldequal 960.) The term ForwardTranslationSpeed(t) refers to the speed ofthe translation along the direction of the optical axis (which may beobtained, for example, by an accelerometer or other component of amotion sensor 240), and Distance(AI) refers to a distance of the targetat the location of the area of interest, which may be obtained, forexample, from the focus position.

As noted above, terms (2) and (3) are values that indicate the amount ofstretch/compression speed at the area of interest at time t during aframe to compensate for translation in a direction along the opticalaxis at time t. Because this is additive to other types of OIS, such as2-axis OIS or 4-axis OIS, these terms may be added to existing x and yOIS lens shift. That said, because terms (2) and (3) are in units ofpixels/s, they may need to first be converted to radians/s by dividingover the focal length of the camera assembly 200 when added to existingOIS lens shift.

For example, terms (2) and (3) may be added to 4-axis OIS lens shift toprovide lens shift for “5-axis” OIS by calculating adjusted x and y OISlens shift as follows:

$\begin{matrix}{{{Xgyr{o_{adj}(t)}} = {{Xgyr{o(t)}} + \frac{{XTrans}lationSpee{d(t)}}{{Distance}({AI})} + \frac{{AI}_{Xspeed}(t)}{FL}}},} & (4)\end{matrix}$for OIS lens shift in a direction along the x axis, and

$\begin{matrix}{{{{Ygyro}_{adj}(t)} = {{{Ygyro}(t)} + \frac{{YTrans}lationSpee{d(t)}}{{Distance}({AI})} + \frac{{AI}_{{Yspeed}{(t)}}}{FL}}},} & (5)\end{matrix}$for OIS lens shift in a direction along the y axis. The terms Xgyro(t)and Ygyro(t) are OIS lens shift values accounting for blur in the x andy directions due to yaw and pitch movements, thus these terms providefor 2-axis OIS. The additional terms XTranslationSpeed(t)/Distance(AI)and YTranslationSpeed(t)/Distance(AI) respectively account fortranslation in directions along x and y axes, providing 4-axis OIS whenadded to the terms for 2-axis OIS lens shift. (Here, the termsXTranslationSpeed(t) and YTranslationSpeed(t) are measurements of speedin directions along x and y axes, respectively, at time t.) Thus, addingthe terms AI_(Xspeed)(t)/FL and AI_(Yspeed)(t)/FL to the 4-axis OIS lensshift, providing OIS compensation due to movement in a direction alongthe optical axis, results in 5-axis OIS.

According to some embodiments, OIS lens shift adjustment in this mannercan be applied to multiple areas of interest within an image by“tightly” synchronizing lens movement with a frame such that the lens210 moves to optimize OIS compensation differently for different rows ofthe image.

FIG. 5 is a frame graph showing how the image sensor 222 captures animage frame by first subjecting rows to an exposure period, then readingout rows, row by row, during a readout period. As with other figuresherein, FIG. 5 shows an example having 1080 rows (e.g., high definition(HD) video quality), although alternative embodiments may vary in thenumber of rows each frame has. The frame 500 is captured by a cameraassembly having a rolling shutter. Therefore, the frame 500 begins withan exposure period followed by a readout period. More specifically,after exposure values for photosensitive sensor elements (image sensorpixels) in a row are reset, the row is subject to an exposure period,after which exposure values for the sensor elements are read andprocessed with downstream image processing circuitry. This exposure andreadout occurs row-by-row, starting with row 1. Thus, the frame 500begins with the exposure of the first row (row 1) and ends with thereadout of the exposure values of the last row (row 1080).

It can be noted that, as used herein, the term “readout period” mayrefer to the time it takes to read the rows of the frame, which may besmaller than native amount of rows of the image sensor 222. For example,in the case of video cropping or digital zoom, only a subset of the rowsof an image sensor 222 may be read. And thus, the readout periodcorresponds to the period of time in which the subset of rows is read.Thus, as used herein, the term “readout period” may refer to this“effective” readout period.

In this example, the frame 500 is split into two subframes, Subframe 1and Subframe 2, as illustrated, to provide OIS compensation for opticalaxis translation for two areas of interest at two locations within theimage. The first area of interest is located within the first half ofthe rows, and the second area of interest is located in the second halfof the rows. It will be understood, however, that subframes may beunequally divided such that one subframe is larger than the other.Moreover, a frame may be divided into more than two subframes toaccommodate more than two areas of interest. In FIG. 5, OIS lens shiftmay be tuned during Subframe 1 in accordance with expressions (4) and(5) to provide OIS compensation due to optical axis translation in theresulting image for a first area of interest, and then “retuned” duringSubframe 2 to provide OIS compensation for optical axis translation fora second area of interest.

FIG. 6 illustrates the blur in the resulting image 600. As can be seen,optical axis translation blur is reduced in a first half of the image600 with respect to a first area of interest, AI1 610, and also reducedin a second half of the image 600 with respect to a second area ofinterest, AI2 620. As previously noted, this concept may be extended toprovide optical axis translation compensation for images having any aplurality of areas of interest. However, because sensor elements of theimage sensor 222 are read out a row at a time, and because adjustmentsto the position of the lens 210 in the xy plane apply correctionuniformly across the row, optical axis translation compensation may onlybe applied to areas of interest occurring on different rows of the image600. As previously noted, areas of interest may be identified using anyof a variety of techniques, including manual user input and/or an objectdetection algorithm (e.g., a face-detection executed by the electronicdevice 100).

According to some embodiments, these concepts may be extended not onlyto multiple areas interest, but to every row in an image. That is,rather than retuning the OIS lens shift for a small number of subframes,embodiments may continually retune the OIS lens shift, as rows of theimage sensor 222 are being exposed, to provide OIS compensation foroptical axis translation for every row. FIGS. 6 and 7 help illustratehow this may be done.

FIG. 7 is a frame graph, similar to FIG. 6, illustrating exposure andreadout of sensor rows over the course of a frame 700. Here, however,rather than dividing the frame 700 in subframes and calculating anadjusted OIS shift using expressions (4) and (5) for each subframe, OISmay be retuned on a per-row basis, depending on which row is beingexposed at a given time t. This causes the adjustment in OIS shift tovary over time, optimizing optical axis translation compensation as eachrow is exposed. FIG. 7 also superimposes the value of a time-dependentfunction used, Weight(t) 710, over the frame 700 to illustrate thechange in value of Weight(t) over time. As can be seen, the Weight(t)may comprise a linear function of time.

As an example, the y-axis OIS lens shift as calculated in expression (5)may be modified as follows:

$\begin{matrix}{{{Ygyr{o_{adj}(t)}} = {{Ygyr{o(t)}} + \frac{{YTrans}lationSpee{d(t)}}{Distance} + \frac{ZTranslationSpe{{ed}(t)}*{{Weight}(t)}}{Distance}}},} & (6)\end{matrix}$where Distance is the distance to the target, ZTranslation Speed(t) is aspeed of translation in a direction along the optical axis, andWeight(t) is an approximately linear function of time as follows:

$\begin{matrix}{{{Weight}(t)} = {\frac{{{Row}(t)} - {MidRow}}{FL}.}} & (7)\end{matrix}$Here, Row(t) is the value of the row exposed at time t.

In the expression (6), the term Distance may be a static term that doesnot vary across over the course of the frame 700. It may be chosen, forexample, based on a distance to a target, and may be determined by anautofocus algorithm, for example. However, according to someembodiments, the term Distance may be replaced with a time-dependentterm, Distance(t), that varies over the course of the frame 700 based onwhich row is exposed at time t. That is:Distance(t)=Distance(Row(t)).  (8)

This optimal distance for each row can be computed from the PhaseDetection Auto Focus (PDAF)-derived depth map and/or other autofocusrelated technologies, for example. Some embodiments may utilize activemethods such as time of light or structured light. Some embodiments mayuse passive methods, such as depth from a stereoscopic (for example,dual) camera. Indirect methods additionally or alternatively may beused, such as mapping the current lens position to a distance (whereaccuracy may depend on the autofocus actuator quality and calibration)or using a PDAF-derived depth map (where it is possible to estimatebefore and/or after focus on different regions, and use the datadifferently (e.g., optimize OIS translation compensation accounting forvariance in distances in the PDAF-derived depth map) compared to thecurrent lens position). According to some embodiments, the PDAF depthmap may also provide prioritization information, indicating which sceneelements are more important than others.

Using expression (8), expression (6) can then be modified as follows:

$\begin{matrix}{{{Ygyr{o_{adj}(t)}} = {{Ygyr{o(t)}} + \frac{{YTrans}lationSpee{d(t)}}{{Distance}(t)} + \frac{ZTranslationSpe{{ed}(t)}*{{Weight}(t)}}{{Distance}(t)}}}.} & (9)\end{matrix}$

The synchronization of OIS with the frame can allow for thedetermination of which Row(t) is exposed at any time, t, in the frame.Thus, as opposed to FIG. 5, in which the OIS lens shift was synchronizedwith the frame capture to allow the OIS lens shift to be modifiedhalfway through the frame, the embodiment illustrated in FIG. 7 is more“tightly” synchronized to the exposure of individual rows throughout theframe. As indicated in FIG. 7, OIS lens shift adjustment may be appliedapproximately halfway through the exposure period of each row. In otherwords, Row(t) may be calculated to be a row approximately halfwaythrough its exposure period at time t, although embodiments may vary.

To provide such time synchronization, the controller 230 can obtain, viaan interrupt or other data related to frame capture, informationregarding when the frame begins, and what the exposure and readoutperiods are for the frame. Where interrupts are unavailable to thecontroller 230, the controller 230 may predict or calculate thebeginning of a frame based on single frame timing. Such frame predictionin video can remain accurate for minutes because frame period istypically very stable in video.

With this information, and information regarding the number of rows inthe sensor, the OIS can apply lens shift adjustment as shown. For animage sensor having 540 rows and an FL of 1500 pixels, the value ofWeight(t) can vary from approximately −0.36 to 0.36. (Again, whether thevalue of Weight(t) is positive or negative for a given row may depend onthe sign conventions used.)

FIG. 8 illustrates how the OIS motion blur compensation for optical (z)axis translation performed in FIG. 7 can reduce or eliminate motion blurin a direction along the y-axis of the resulting image 800. In FIG. 8, acomparison between uncompensated blur (“W/O Compensation”) andcompensation for optical axis translation as described herein (“Y BlurGone”). As can be seen, compensation for optical axis translationgreatly reduces motion blur in a direction along the y-axis.

Thus, tightly synchronized OIS lens shift adjustment as shown in FIG. 7and described above can greatly reduce motion blur due to optical axistranslation by eliminating the y component of motion blur. That said,the vertical component of the motion blur remains. This is becauseexposure value of sensor elements the image sensor 222 are read out arow to time, allowing embodiments to optimize for each row, but not eachcolumn. Although optimization is not additionally adjusted for eachcolumn, providing OIS lens shift adjustment in this way can bebeneficial in many applications. If nothing else, it reduces the amountof motion blur due to optical axis translation by expanding an area thatis relatively unaffected by motion blur from a center circular area (orvanishing point, e.g., as shown in FIG. 3) to a column that stretchesfrom the top to the bottom of the image (as shown in FIG. 8).

In some embodiments, special considerations may be made to help ensurethe blur compensation techniques herein are not used in cases wheredifferent rows of the image sensor 222 may be exposed at the same timefor frames captured in succession, such as in video, or where anexposure period is significantly large relative to a readout period.

FIG. 9 illustrates a frame graph to help illustrate this point. In FIG.9, two frames, first frame 910 and second frame 920, are captured insuccession. As can be seen, the exposure period, readout period, andframe period for the frames 910, 920 are such that there is no overlapin time between the frames. In other words (as shown by the exposed rowsat time T1 and exposed rows at time T2, for example) at no time woulddifferent sets of rows of the image sensor be exposed for the differentframes 910, 920, and at any given time, a relatively small percentage ofthe rows of the image sensor 222 are exposed. This may not be the casefor frames having a relative long exposure period.

FIG. 10 illustrates a frame graph to help illustrate this point. Similarto FIG. 9, FIG. 10 shows two frames, First Frame 1010 and Second Frame1020, captured in succession. Here, however, the exposure period islengthened (e.g., due to low light and/or other conditions making anextended exposure period desirable). This may cause one or two issues toarise.

A first issue is that a long exposure period relative to the readoutperiod can result in a large amount of rows exposed at a given time in aframe. At time T1, for example, nearly all rows are exposed. This canreduce the effectiveness of embodiments of tightly-synchronized OIS rollcompensation techniques described herein, in which the lens position isbased on the exposed rows, because more rows are exposed at once. Thisissue may arise in successively-captured frames (as shown) or singleframes.

A second issue is that lengthening an exposure period can cause framesto overlap. For example, it may be desirable in some instances tomaintain a certain frame rate (e.g., 30 frame per second (fps) video)even with longer exposure times. As such, there may be instances, suchas at time T2 in FIG. 10, in which different rows of the image sensor222 are exposed at the same time for different frames, presenting aproblem for tightly-synchronized OIS roll compensation, where lensposition is dependent on an exposed row. The two different sets ofexposed rows would require the lens to be in two different positions.

To alleviate these issues, embodiments may employ the controller 230 orother processing unit to monitor the exposure period, readout period,and frame period of successively-captured frames. Where it is determinedthat frames might overlap and/or where an exposure period exceeds athreshold length relative to the length of the readout period (e.g., 50%or more, resulting in a large number of rows exposed simultaneously),the controller 230 or other processing unit may then decrease the lengthof the exposure period such that the length of the exposure period to adesired length (e.g., below the threshold length relative to the lengthof the readout period) and increase an ISO of the image sensor to atleast partially compensate for the decrease in the length of theexposure period. Additionally or alternatively, if the camera assemblyis capable of adjusting the aperture (i.e., reducing the focal ratio, orf-stop), the controller 230 or other processing unit may increase theaperture of the camera assembly to at least partially compensate for thedecrease in the length of the exposure period. Additionally oralternatively, if the camera assembly is capable of adjusting the framerate (i.e., increasing the frame period), the controller 230 or otherprocessing unit may reduce the frame rate to ensure that frames do notoverlap. Such adjustments may be performed, for example, in a still shotpreview mode.

FIG. 11 is a flow diagram of a method 1100 of providing synchronized OISin a camera assembly having an adjustable lens, according to anembodiment. Means for performing the method may include one or morecomponents of the camera assembly, such as the camera assembly 200 inFIG. 2. In particular, the functionality described in the blocks ofmethod 1100 may be performed by, for example, a processing unit (e.g.,controller 230) capable of moving a lens in response to motion sensordata.

At block 1110, the functionality includes, for an image sensor of thecamera assembly with sensor elements disposed in rows oriented in adirection along an first axis and columns oriented in a direction alonga second axis, obtaining information indicative of the beginning of aframe of the image sensor, wherein the frame comprises a period of timehaving an exposure period followed by a readout period in which exposurevalues are read from the sensor elements row by row. As previouslyindicated, embodiments herein may be applied to camera assemblies havinga rolling shutter in which sensor elements are read row by row, whererows are aligned in a direction along the first access (e.g., thex-axis), and columns are aligned in a direction along the second axis(e.g., the y-axis). Moreover, as noted elsewhere herein, obtaininginformation indicative of the beginning of a frame may compriseobtaining an interrupt or other direct indicator of the beginning of aframe. Additionally or alternatively, the beginning of a frame may beobtained from a calculation based on indirect data indicative of thebeginning of a frame, such as the beginning of video capture.

At block 1120, the functionality includes obtaining at least onedistance value indicative of at least one distance of the image sensorto at least one object viewable by the image sensor. Functionality canvary, as described in the embodiments above. For example, in FIGS. 4 and6, one or more areas of interest may be identified, and a distance toeach of the areas of interest may be determined, e.g. via an autofocusmodule of the camera assembly and/or electronic device. In someembodiments, the areas of interest may be prioritized, and thisprioritization may be used in determining how to position the lensduring the frame. For example, in alternative embodiments, the method1100 may further comprise obtaining, from the autofocus module, at leastone priority value related to the at least one distance value, where,for each time of the plurality of times during the frame, the respectiveposition is additionally at least partially based on the at least onepriority value related to the at least one distance value. Additionallyor alternatively, the one or more areas of interest may be identifiedfrom user input. In some embodiments, one or more preliminary images maybe captured and/or other sensor data may be obtained, from which areasof interest, distance, or both, may be determined.

At block 1130, the functionality comprises, for each time of a pluralityof times during the frame, performing functions illustrated in blocks1130-a, 1130-b, and 1130-c. At block 1130-a, the functionality includesobtaining respective movement data indicative of translation of thecamera assembly in a direction along a third axis orthogonal to thefirst axis and the second axis, the translation corresponding to therespective time. At block 1130-b, the functionality includes,determining a respective value of a time-dependent function based atleast in part on the information indicative of the beginning of theframe, where the respective value of the time-dependent function isdependent on the respective time in relation to the frame. And at block1130-c, the functionality includes, causing the adjustable lens to bemoved to a respective position relative to the image sensor during theframe, where the respective position in a direction along the firstaxis, in a direction along the second axis, or both, is at leastpartially based on the respective movement data, the respective value ofthe time-dependent function, and the at least one distance value.

As noted in the embodiments above, the time-dependent function may be atleast partially based on a number of rows of sensor elements of theimage sensor. As previously noted, the time-dependent function maycomprise a linear time-dependent weight (e.g., Weight(t)).

Additionally or alternatively, a value of distance from the image sensorto an object of an area of interest or row may also be time-dependent(e.g., Distance(t)), although alternative embodiments may vary. Thus, insome embodiments, the information indicative of the at least onedistance value of the image sensor to the at least one object, obtainedat block 1120, comprises information indicative of a plurality ofdistance values, where each distance value is indicative of a distanceof the image sensor to the least one object. For each time of theplurality of times during the frame, the at least one distance value canthen comprise a respective distance value of the plurality of distancevalues.

As indicated previously in the description, and illustrated in FIGS.9-10, the method may further comprise, prior to the beginning of theframe, determining the length of the exposure period and a length of thereadout period. This can help determine whether a number of rows exposedat a time during the frame may be too numerous for the synchronized OISstabilization to work effectively. The method may include monitoringthese values and proceeding once it is determined that the they fallwithin a favorable range. Thus, embodiments of the method 1100 mayadditionally or alternatively comprise determining that the length ofthe exposure period does not exceed a threshold length in relation tothe length of the readout period, where determining the respective valueof the time-dependent function for each time of the plurality of timesduring the frame is in response to determining that the length of theexposure period does not exceed the threshold length of the readoutperiod. As previously noted, this threshold length may vary, dependingon desired functionality. In some embodiments, the threshold length ofthe exposure period may be approximately 50% of the length of thereadout period. In other embodiments, the threshold may be less thanthis (e.g., 40%, 30%, or less) or greater than this (e.g., 60%, etc.).

In instances where the exposure period exceeds the threshold, someembodiments may allow for the reduction of this exposure period and theincrease of the ISO and/or widening of an aperture to keep the sensorelements sufficiently exposed. For example, in some embodiments, priorto the beginning of the frame, the method 1100 may further includedetermining that a length of the exposure period exceeds a thresholdlength in relation to a length of the readout period and, in response todetermining that the length of the exposure period exceeds the thresholdlength of the readout period, decreasing the length of the exposureperiod such that the length of the exposure period does not exceed thethreshold length of the readout period, and at least partiallycompensating for the decrease in the length of the exposure period byincreasing an ISO of the image sensor, widening an aperture of thecamera assembly configured to adjust an amount of light to which theimage sensor is exposed, or both.

It will be apparent to those skilled in the art that substantialvariations may be made in accordance with specific requirements. Forexample, customized hardware might also be used, and/or particularelements might be implemented in hardware, software (including portablesoftware, such as applets, etc.), or both. Further, connection to othercomputing devices such as network input/output devices may be employed.

With reference to the appended figures, components that can includememory (e.g., controller 230) can include non-transitorymachine-readable media. The term “machine-readable medium” and“computer-readable medium” as used herein, refer to any storage mediumthat participates in providing data that causes a machine to operate ina specific fashion. In embodiments provided hereinabove, variousmachine-readable media might be involved in providing instructions/codeto processing units and/or other device(s) for execution. Additionallyor alternatively, the machine-readable media might be used to storeand/or carry such instructions/code. In many implementations, acomputer-readable medium is a physical and/or tangible storage medium.Such a medium may take many forms, including but not limited to,non-volatile media, volatile media, and transmission media. Common formsof computer-readable media include, for example, magnetic and/or opticalmedia, any other physical medium with patterns of holes, a RAM, a PROM,EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier waveas described hereinafter, or any other medium from which a computer canread instructions and/or code.

The methods, systems, and devices discussed herein are examples. Variousembodiments may omit, substitute, or add various procedures orcomponents as appropriate. For instance, features described with respectto certain embodiments may be combined in various other embodiments.Different aspects and elements of the embodiments may be combined in asimilar manner. The various components of the figures provided hereincan be embodied in hardware and/or software. Also, technology evolvesand, thus, many of the elements are examples that do not limit the scopeof the disclosure to those specific examples.

It has proven convenient at times, principally for reasons of commonusage, to refer to such signals as bits, information, values, elements,symbols, characters, variables, terms, numbers, numerals, or the like.It should be understood, however, that all of these or similar terms areto be associated with appropriate physical quantities and are merelyconvenient labels. Unless specifically stated otherwise, as is apparentfrom the discussion above, it is appreciated that throughout thisSpecification discussions utilizing terms such as “processing,”“computing,” “calculating,” “determining,” “ascertaining,”“identifying,” “associating,” “measuring,” “performing,” or the likerefer to actions or processes of a specific apparatus, such as a specialpurpose computer or a similar special purpose electronic computingdevice. In the context of this Specification, therefore, a specialpurpose computer or a similar special purpose electronic computingdevice is capable of manipulating or transforming signals, typicallyrepresented as physical electronic, electrical, or magnetic quantitieswithin memories, registers, or other information storage devices,transmission devices, or display devices of the special purpose computeror similar special purpose electronic computing device.

Terms, “and” and “or” as used herein, may include a variety of meaningsthat also is expected to depend at least in part upon the context inwhich such terms are used. Typically, “or” if used to associate a list,such as A, B, or C, is intended to mean A, B, and C, here used in theinclusive sense, as well as A, B, or C, here used in the exclusivesense. In addition, the term “one or more” as used herein may be used todescribe any feature, structure, or characteristic in the singular ormay be used to describe some combination of features, structures, orcharacteristics. However, it should be noted that this is merely anillustrative example and claimed subject matter is not limited to thisexample. Furthermore, the term “at least one of” if used to associate alist, such as A, B, or C, can be interpreted to mean any combination ofA, B, and/or C, such as A, AB, AA, AAB, AABBCCC, etc.

Having described several embodiments, various modifications, alternativeconstructions, and equivalents may be used without departing from thespirit of the disclosure. For example, the above elements may merely bea component of a larger system, wherein other rules may take precedenceover or otherwise modify the application of the various embodiments.Also, a number of steps may be undertaken before, during, or after theabove elements are considered. Accordingly, the above description doesnot limit the scope of the disclosure.

What is claimed is:
 1. A method of providing synchronized optical imagestabilization in a camera assembly having an adjustable lens, the methodcomprising: for an image sensor of the camera assembly with sensorelements located in a plane, obtaining information indicative of abeginning of a period of time in which a frame is captured by the imagesensor; obtaining at least one distance value indicative of at least onedistance of the image sensor to at least one object viewable by theimage sensor; and during the period of time: obtaining movement dataindicative of translation of the camera assembly in a direction along anaxis orthogonal to the plane during the period of time; determining anexposed portion of sensor elements of the image sensor based at least inpart on the information indicative of the beginning of the period oftime; and causing the adjustable lens to be moved to a position relativeto the image sensor, wherein the position is at least partially based onthe movement data, the exposed portion, and the at least one distancevalue.
 2. The method of claim 1, wherein the sensor elements aredisposed in rows located in the plane.
 3. The method of claim 2, whereindetermining the exposed portion of sensor elements is based on a numberof rows of sensor elements of the image sensor.
 4. The method of claim1, wherein: the information indicative of the at least one distancevalue of the image sensor to the at least one object comprisesinformation indicative of a plurality of distance values, wherein eachdistance value is indicative of a distance of the image sensor to the atleast one object; and the at least one distance value comprises adistance value of the plurality of distance values.
 5. The method ofclaim 1, wherein obtaining the at least one distance value comprisesobtaining the at least one distance value from an autofocus module. 6.The method of claim 5, further comprising obtaining, from the autofocusmodule, at least one priority value related to the at least one distancevalue, wherein the position is additionally at least partially based onthe at least one priority value related to the at least one distancevalue.
 7. The method of claim 1, wherein the period of time comprises anexposure period followed by a readout period in which exposure valuesare read from the sensor elements, the method further comprising, priorto the beginning of the period of time, determining a length of theexposure period and a length of the readout period.
 8. The method ofclaim 7, further comprising: determining that the length of the exposureperiod does not exceed a threshold length in relation to the length ofthe readout period; wherein determining the exposed portion of sensorelements is in response to determining that the length of the exposureperiod does not exceed the threshold length of the readout period. 9.The method of claim 7, further comprising: prior to the beginning of theperiod of time, determining that a length of the exposure period exceedsa threshold length in relation to a length of the readout period; and inresponse to determining that the length of the exposure period exceedsthe threshold length of the readout period, decreasing the length of theexposure period such that the length of the exposure period does notexceed the threshold length of the readout period, and at leastpartially compensating for the decrease in the length of the exposureperiod by: increasing an ISO of the image sensor, widening an apertureof the camera assembly configured to adjust an amount of light to whichthe image sensor is exposed, or both.
 10. A camera assembly with opticalimage stabilization, the camera assembly comprising: a controllerconfigured to be communicatively coupled with: an image sensor withsensor elements located in a plane; one or more actuators configured tomove an adjustable lens that focuses light onto the image sensor; and amotion sensor; wherein the controller is configured to: obtaininformation indicative of a beginning of a period of time in which aframe is captured by the image sensor; obtain at least one distancevalue indicative of at least one distance of the image sensor to atleast one object viewable by the image sensor; and during the period oftime: obtain movement data from the motion sensor indicative oftranslation of the camera assembly in a direction along an axisorthogonal to the plane during the period of time; determine an exposedportion of sensor elements of the image sensor based at least in part onthe information indicative of the beginning of the period of time; andcause the one or more actuators to move the adjustable lens to aposition relative to the image sensor, wherein the position is at leastpartially based on the movement data, the exposed portion, and the atleast one distance value.
 11. The camera assembly of claim 10, whereinthe sensor elements are disposed in rows located in the plane.
 12. Thecamera assembly of claim 11, wherein determining the exposed portion ofsensor elements is based on a number of rows of sensor elements of theimage sensor.
 13. The camera assembly of claim 10, wherein: theinformation indicative of the at least one distance value of the imagesensor to the at least one object comprises information indicative of aplurality of distance values, wherein each distance value is indicativeof a distance of the image sensor to the at least one object, and the atleast one distance value comprises a distance value of the plurality ofdistance values.
 14. The camera assembly of claim 10, wherein thecontroller is configured to obtain the at least one distance value atleast in part by obtaining the at least one distance value from anautofocus module.
 15. The camera assembly of claim 14, wherein thecontroller is further configured to obtain, from the autofocus module,at least one priority value related to the at least one distance value,wherein the position is additionally at least partially based on the atleast one priority value related to the at least one distance value. 16.The camera assembly of claim 10, wherein the period of time comprises anexposure period followed by a readout period in which exposure valuesare read from the sensor elements, and the controller is furtherconfigured to, prior to the beginning of the period of time, determine alength of the exposure period and a length of the readout period. 17.The camera assembly of claim 16, wherein the controller is furtherconfigured to determine that the length of the exposure period does notexceed a threshold length in relation to the length of the readoutperiod; and wherein the controller is configured to determine theexposed portion of sensor elements in response to determining that thelength of the exposure period does not exceed the threshold length ofthe readout period.
 18. The camera assembly of claim 16, wherein thecontroller is further configured to: prior to the beginning of theperiod of time, determine that a length of the exposure period exceeds athreshold length in relation to a length of the readout period; and inresponse to determining that the length of the exposure period exceedsthe threshold length of the readout period, decrease the length of theexposure period such that the length of the exposure period does notexceed the threshold length of the readout period, and at leastpartially compensating for the decrease in the length of the exposureperiod by: increasing an ISO of the image sensor, widening an apertureof the camera assembly configured to adjust an amount of light to whichthe image sensor is exposed, or both.
 19. An apparatus comprising: meansfor obtaining, for an image sensor of the apparatus with sensor elementslocated in a plane, information indicative of a beginning of a period oftime in which a frame is captured by the image sensor; means forobtaining at least one distance value indicative of at least onedistance of the image sensor to at least one object viewable by theimage sensor; and means for, during the period of time: obtainingmovement data indicative of translation of the apparatus in a directionalong an axis orthogonal to the plane during the period of time;determining an exposed portion of sensor elements of the image sensorbased at least in part on the information indicative of the beginning ofthe period of time; and causing an adjustable lens to be moved to aposition relative to the image sensor, wherein the position is at leastpartially based on the movement data, the exposed portion, and the atleast one distance value.
 20. The apparatus of claim 19, wherein thesensor elements are disposed in rows located in the plane.
 21. Theapparatus of claim 20, wherein determining the exposed portion of sensorelements is based on a number of rows of sensor elements of the imagesensor.
 22. The apparatus of claim 19, wherein: the informationindicative of the at least one distance value of the image sensor to theat least one object comprises information indicative of a plurality ofdistance values, wherein each distance value is indicative of a distanceof the image sensor to the at least one object, and the at least onedistance value comprises a distance value of the plurality of distancevalues.
 23. The apparatus of claim 19, wherein the means for obtainingthe at least one distance value comprises means for obtaining the atleast one distance value from an autofocus module.
 24. The apparatus ofclaim 23, further comprising means for obtaining, from the autofocusmodule, at least one priority value related to the at least one distancevalue, wherein the position is additionally at least partially based onthe at least one priority value related to the at least one distancevalue.
 25. The apparatus of claim 19, wherein the period of timecomprises an exposure period followed by a readout period in whichexposure values are read from the sensor elements, the apparatus furthercomprising means for determining, prior to the beginning of the periodof time, a length of the exposure period and a length of the readoutperiod.
 26. The apparatus of claim 25, further comprising: means fordetermining that the length of the exposure period does not exceed athreshold length in relation to the length of the readout period;wherein the means for determining the exposed portion of sensor elementsare configured to do so in response to determining that the length ofthe exposure period does not exceed the threshold length of the readoutperiod.
 27. The apparatus of claim 25, further comprising: means fordetermining, prior to the beginning of the period of time, that a lengthof the exposure period exceeds a threshold length in relation to alength of the readout period; and means for decreasing the length of theexposure period, in response to determining that the length of theexposure period exceeds the threshold length of the readout period, suchthat the length of the exposure period does not exceed the thresholdlength of the readout period, and at least partially compensating forthe decrease in the length of the exposure period by: increasing an ISOof the image sensor, widening an aperture of the apparatus configured toadjust an amount of light to which the image sensor is exposed, or both.28. A non-transitory computer-readable medium having instructionsembedded thereon for providing synchronized optical image stabilizationin a camera assembly having an adjustable lens, the instructions, whenexecuted by one or more processing units, cause the one or moreprocessing units to: for an image sensor of the camera assembly withsensor elements located in a plane, obtain information indicative of abeginning of a period of time in which a frame is captured by the imagesensor; obtain at least one distance value indicative of at least onedistance of the image sensor to at least one object viewable by theimage sensor; and during the period of time: obtain movement dataindicative of translation of the camera assembly in a direction along anaxis orthogonal to the plane during the period of time; determine anexposed portion of sensor elements of the image sensor based at least inpart on the information indicative of the beginning of the period oftime; and cause the adjustable lens to be moved to a position relativeto the image sensor, wherein the position is at least partially based onthe movement data, the exposed portion, and the at least one distancevalue.
 29. The non-transitory computer-readable medium of claim 28,wherein the sensor elements are disposed in rows located in the plane.30. The non-transitory computer-readable medium of claim 29, whereindetermining the exposed portion of sensor elements is based on a numberof rows of sensor elements of the image sensor.
 31. The non-transitorycomputer-readable medium of claim 28, wherein: the informationindicative of the at least one distance value of the image sensor to theat least one object comprises information indicative of a plurality ofdistance values, wherein each distance value is indicative of a distanceof the image sensor to the at least one object; and the at least onedistance value comprises a distance value of the plurality of distancevalues.
 32. The non-transitory computer-readable medium of claim 28,wherein the instructions that cause the one or more processing units toobtain the at least one distance value further comprise instructionsthat, when executed by one or more processing units, cause the one ormore processing units to obtain the at least one distance value from anautofocus module.
 33. The non-transitory computer-readable medium ofclaim 32, further comprising instructions that, when executed by one ormore processing units, cause the one or more processing units to obtain,from the autofocus module, at least one priority value related to the atleast one distance value, wherein the position is additionally at leastpartially based on the at least one priority value related to the atleast one distance value.
 34. The non-transitory computer-readablemedium of claim 28, wherein the period of time comprises an exposureperiod followed by a readout period in which exposure values are readfrom the sensor elements, the non-transitory computer-readable mediumfurther comprising instructions that, when executed by one or moreprocessing units, cause the one or more processing units to determine,prior to the beginning of the period of time, a length of the exposureperiod and a length of the readout period.
 35. The non-transitorycomputer-readable medium of claim 34, further comprising instructionsthat, when executed by one or more processing units, cause the one ormore processing units to determine that the length of the exposureperiod does not exceed a threshold length in relation to the length ofthe readout period; wherein determining the exposed portion of sensorelements is in response to determining that the length of the exposureperiod does not exceed the threshold length of the readout period. 36.The non-transitory computer-readable medium of claim 34, furthercomprising instructions that, when executed by one or more processingunits, cause the one or more processing units to: prior to the beginningof the period of time, determine that a length of the exposure periodexceeds a threshold length in relation to a length of the readoutperiod; and in response to determining that the length of the exposureperiod exceeds the threshold length of the readout period, decrease thelength of the exposure period such that the length of the exposureperiod does not exceed the threshold length of the readout period, andat least partially compensating for the decrease in the length of theexposure period by: increasing an ISO of the image sensor, widening anaperture of the camera assembly configured to adjust an amount of lightto which the image sensor is exposed, or both.